# Structure-enhanced BERT

StructBERT is a novel model that extends BERT by incorporating linguistic structures into the pre-training process, leveraging two auxiliary tasks to fully utilize word and sentence order structures